{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Semantic Joins with LLM Reasoning\n",
    "This example demonstrates how to use semantic joins powered by large language models (LLMs) in Fenic to perform context-aware data matching and recommendations.\n",
    "\n",
    "Semantic joins enable you to flexibly join tables based on natural language instructions, leveraging LLM reasoning rather than strict key or embedding similarity.\n",
    "\n",
    "In this notebook, you'll see two practical scenarios:\n",
    "\n",
    "- **Matching users to relevant articles**: Join user profiles with articles by reasoning about their interests and the article descriptions.\n",
    "- **Product recommendations**: Suggest products to customers based on their purchase history using semantic relationships between purchased and recommended items.\n",
    "\n",
    "This approach allows for more intuitive and powerful data enrichment, recommendation, and matching workflows, all driven by natural language join instructions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up the fenic session\n",
    "\n",
    "here we configure the session to use a language model for semantic joins. No embeddings are required, only LLM reasoning is used for flexible, context-aware data matching."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import fenic as fc\n",
    "\n",
    "\"\"\"Demonstrate semantic join capabilities using LLM reasoning.\"\"\"\n",
    "# Configure session with language models (no embeddings needed)\n",
    "config = fc.SessionConfig(\n",
    "    app_name=\"semantic_joins\",\n",
    "    semantic=fc.SemanticConfig(\n",
    "        language_models={\n",
    "            \"mini\": fc.OpenAIModelConfig(\n",
    "                model_name=\"gpt-4o-mini\",\n",
    "                rpm=500,\n",
    "                tpm=200_000,\n",
    "            )\n",
    "        }\n",
    "    ),\n",
    ")\n",
    "\n",
    "# Create session\n",
    "session = fc.Session.get_or_create(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Example DataFrames\n",
    "\n",
    "In this section, we define sample user profiles and articles as Python dictionaries and load them into Fenic DataFrames. \n",
    "\n",
    "This data will be used to demonstrate how semantic joins can match users to relevant articles based on their interests and the article descriptions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sample user profiles data\n",
    "users_data = [\n",
    "    {\n",
    "        \"user_id\": \"user_001\",\n",
    "        \"name\": \"Sarah\",\n",
    "        \"interests\": \"I love cooking Italian food and trying new pasta recipes\"\n",
    "    },\n",
    "    {\n",
    "        \"user_id\": \"user_002\",\n",
    "        \"name\": \"Mike\",\n",
    "        \"interests\": \"I enjoy working on cars and fixing engines in my spare time\"\n",
    "    },\n",
    "    {\n",
    "        \"user_id\": \"user_003\",\n",
    "        \"name\": \"Emily\",\n",
    "        \"interests\": \"Gardening is my passion, especially growing vegetables and flowers\"\n",
    "    },\n",
    "    {\n",
    "        \"user_id\": \"user_004\",\n",
    "        \"name\": \"David\",\n",
    "        \"interests\": \"I'm interested in learning about car maintenance and automotive repair\"\n",
    "    }\n",
    "]\n",
    "\n",
    "# Sample content/articles data\n",
    "articles_data = [\n",
    "    {\n",
    "        \"article_id\": \"art_001\",\n",
    "        \"title\": \"Cooking Pasta Recipes\",\n",
    "        \"description\": \"Delicious pasta recipes including spaghetti carbonara and fettuccine alfredo\"\n",
    "    },\n",
    "    {\n",
    "        \"article_id\": \"art_002\",\n",
    "        \"title\": \"Car Engine Maintenance\",\n",
    "        \"description\": \"Essential guide to automobile engine care and troubleshooting\"\n",
    "    },\n",
    "    {\n",
    "        \"article_id\": \"art_003\",\n",
    "        \"title\": \"Gardening for Beginners\",\n",
    "        \"description\": \"Start your garden with basic techniques for growing vegetables and flowers\"\n",
    "    },\n",
    "    {\n",
    "        \"article_id\": \"art_004\",\n",
    "        \"title\": \"Advanced Automotive Repair\",\n",
    "        \"description\": \"Comprehensive automotive repair instructions for experienced mechanics\"\n",
    "    }\n",
    "]\n",
    "\n",
    "# Create DataFrames\n",
    "users_df = session.create_dataframe(users_data)\n",
    "articles_df = session.create_dataframe(articles_data)\n",
    "\n",
    "print(\"User Profiles:\")\n",
    "users_df.select(\"name\", \"interests\").show()\n",
    "print()\n",
    "\n",
    "print(\"Available Articles:\")\n",
    "articles_df.select(\"title\", \"description\").show()\n",
    "print()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Semantic Join: Matching Users to Relevant Articles\n",
    "\n",
    "In this step, we use a semantic join to match each user with articles that align with their interests. \n",
    "\n",
    "The join leverages LLM reasoning and a natural language instruction to determine which articles would be relevant for each user, based on the content of their interests and the article descriptions.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 1: Semantic join to match users with relevant articles\n",
    "print(\"Step 1: Matching users to relevant articles using semantic reasoning...\")\n",
    "print(\"-\" * 70)\n",
    "\n",
    "# Use semantic join to match users with articles based on their interests\n",
    "user_article_matches = users_df.semantic.join(\n",
    "    articles_df,\n",
    "    join_instruction=\"A person with interests '{interests:left}' would be interested in reading about '{description:right}'\"\n",
    ")\n",
    "\n",
    "print(\"User-Article Matches:\")\n",
    "user_article_matches.select(\n",
    "    \"name\",\n",
    "    \"interests\",\n",
    "    \"title\",\n",
    "    \"description\"\n",
    ").show()\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing Data for Product Recommendations\n",
    "\n",
    "In this section, we create sample data for customer purchase history and a product catalog. \n",
    "\n",
    "These are loaded into Fenic DataFrames and will be used to demonstrate how semantic joins can recommend new products to customers based on the relationships between their previous purchases and available products."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 2: Product recommendation system using semantic joins\n",
    "print(\"Step 2: Product recommendation system...\")\n",
    "print(\"-\" * 50)\n",
    "\n",
    "# Sample customer purchase history\n",
    "purchases_data = [\n",
    "    {\n",
    "        \"customer_id\": \"cust_001\",\n",
    "        \"customer_name\": \"Alice\", \n",
    "        \"purchased_product\": \"Professional DSLR Camera\"\n",
    "    },\n",
    "    {\n",
    "        \"customer_id\": \"cust_002\",\n",
    "        \"customer_name\": \"Bob\",\n",
    "        \"purchased_product\": \"Gaming Laptop\"\n",
    "    },\n",
    "    {\n",
    "        \"customer_id\": \"cust_003\",\n",
    "        \"customer_name\": \"Carol\",\n",
    "        \"purchased_product\": \"Yoga Mat\"\n",
    "    },\n",
    "    {\n",
    "        \"customer_id\": \"cust_004\",\n",
    "        \"customer_name\": \"Dan\",\n",
    "        \"purchased_product\": \"Coffee Maker\"\n",
    "    }\n",
    "]\n",
    "\n",
    "# Sample product catalog for recommendations\n",
    "products_data = [\n",
    "    {\n",
    "        \"product_id\": \"prod_001\",\n",
    "        \"product_name\": \"Camera Lens Kit\",\n",
    "        \"category\": \"Photography\"\n",
    "    },\n",
    "    {\n",
    "        \"product_id\": \"prod_002\",\n",
    "        \"product_name\": \"Tripod Stand\", \n",
    "        \"category\": \"Photography\"\n",
    "    },\n",
    "    {\n",
    "        \"product_id\": \"prod_003\",\n",
    "        \"product_name\": \"Gaming Mouse\",\n",
    "        \"category\": \"Gaming\"\n",
    "    },\n",
    "    {\n",
    "        \"product_id\": \"prod_004\",\n",
    "        \"product_name\": \"Mechanical Keyboard\",\n",
    "        \"category\": \"Gaming\"\n",
    "    },\n",
    "    {\n",
    "        \"product_id\": \"prod_005\",\n",
    "        \"product_name\": \"Yoga Blocks\",\n",
    "        \"category\": \"Fitness\"\n",
    "    },\n",
    "    {\n",
    "        \"product_id\": \"prod_006\",\n",
    "        \"product_name\": \"Exercise Resistance Bands\",\n",
    "        \"category\": \"Fitness\"\n",
    "    },\n",
    "    {\n",
    "        \"product_id\": \"prod_007\",\n",
    "        \"product_name\": \"Coffee Beans Premium Blend\",\n",
    "        \"category\": \"Food & Beverage\"\n",
    "    },\n",
    "    {\n",
    "        \"product_id\": \"prod_008\",\n",
    "        \"product_name\": \"French Press\",\n",
    "        \"category\": \"Food & Beverage\"\n",
    "    }\n",
    "]\n",
    "\n",
    "# Create DataFrames\n",
    "purchases_df = session.create_dataframe(purchases_data)\n",
    "products_df = session.create_dataframe(products_data)\n",
    "\n",
    "print(\"Customer Purchase History:\")\n",
    "purchases_df.select(\"customer_name\", \"purchased_product\").show()\n",
    "print()\n",
    "\n",
    "print(\"Available Products for Recommendation:\")\n",
    "products_df.select(\"product_name\", \"category\").show()\n",
    "print()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Semantic Join: Generating Product Recommendations\n",
    "\n",
    "In this step, we use a semantic join to recommend products to customers based on their previous purchases. \n",
    "\n",
    "The join uses LLM reasoning and a natural language instruction to identify which products in the catalog are likely to interest each customer, given what they have already bought."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use semantic join for product recommendations\n",
    "recommendations = purchases_df.semantic.join(\n",
    "    products_df,\n",
    "    join_instruction=\"A customer who bought '{purchased_product:left}' would also be interested in '{product_name:right}'\"\n",
    ")\n",
    "\n",
    "print(\"Product Recommendations:\")\n",
    "recommendations.select(\n",
    "    \"customer_name\",\n",
    "    \"purchased_product\", \n",
    "    \"product_name\",\n",
    "    \"category\"\n",
    ").show()\n",
    "print()\n",
    "\n",
    "# Clean up\n",
    "session.stop()\n",
    "print(\"Session complete!\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
